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Achievement Profiles for Elementary Schools 



Abstract 

With the growing interest in educational accountability and outcomes based 
education there is an increasing emphasis upon testing. Ihis study proposes the 
development of school achievement profiles as a tool to assist with the interpretation 
and implementation of test results for targeting iriStractional improvement efforts. 
The test scores from a set of peer institutions are employed as a basis for 
determiiung the relative achievement of individual schools. Standardized residuals 
from a regression analysis with socioeconomic measures as independent variables 
are used to indicate the relative strengths and weaknesses of student achievement 
within various grade levels and subject areas. The profiles are designed to assist in 
giving direction for allocation of instructional improvement efforts to attain a 
balance in student achievement 



Achievement Profiles for Elementary Schools 



In recent years there has been a growing interest in accountability and testing in 
schools. This has lead to the development of statewide testing programs in many 
states. Missouri like other states has gone through a lengthy process of identifying 
the instructional goals for each grade level and subject area. These goals were then 
broken down into key skills which the students are expected to attain. The 
Missouri Mastery Achievement Test (MMAT) is a large battery of tests developed to 
measure student progress on these skills in reading, mathematics, science and social 
studies for grades two through ten. Only reading and mathematics are tested in the 
second grade. Almost all schools in the state administer the test in the spring of 
each year. 

Issues Concerning Statewide Testing 

t 

The administration of statewide achievement tests has been received with mixed 
feelings on the part of many educators. There are critics who say the tests are a tool 
to gain control of education and the process of instruction at the state level 
(Airasian, 1987). With the broad scale testing we are indirectly defining the 
curriculum in terms of the tests which leads to a test driven curriculum (Madaus, 
1988; Livingston, Castle & Nations, 1989). With measurement driven instruction 
there are several side effects such as standardizing the curriculum among schools. 

If the tests are viewed as high-stakes tests teachers tend to teach to the test (Madaus, 
1989). Some educational goals are easier to test for than others. Because of ease in 
testing outcomes based education may shift the teaching efforts to the things that are 
easy to test. 
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Others might say the purpose of the testing it to evaluate the students, teachers and 
schools. If a primary goal is evaluation of teachers this tends to put undue pressure 
on teachers and leads to deprofessionalization of teachers (Livingston, Castle and 
Nations, 1989). Guskey and Kifer (1990) found that ranking the school districts of 
Kentucky based upon different subject area test scores leads to very inconsistent 
ranking of the districts. Mandeville and Anderson (1987) used multiple regression 
techniques to adjust test scores for the socioeconomic levels of students and used 
the standardized residuals as a school effectiveness index (SEI). The limited cross- 
subject stability and cross-year stabilty of the SEI's lead to questions concerning the 
value of test scores as a primary measure for evaluating schools (Mandeville, 

1988). However, there is enough internal consistency of the SEI’s to give 
some general indication of the level of relative achievement within the schools. 

The original intention of the development of instructional goals and the testing 
effort associated with the MMAT was the improvement of instruction. Raybom 
(1988) points out that the value of testing for instructional improvement depends 
upon how we learn to interpret and use the results of the testing effort. This 
research effort is concerned with the interpretation and application of the test 
results. 

Intrepretation of test scores at the school level 

After administering the tests we end up with a large set of numbers which we do 
not use very well. There are all of the individual student scores by subject area. 
Then we have average test scores by school and eventually there are state averages 
for each subject area and grade level. This leads to the question of how is an 
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effective way to organize the results of the testing effort The testing needs to give 
us some meaningful direction as to where to allocate our instmctional improvement 
efforts. There are no absolutes in educational achievement and so all of the test 
scores are relative. Hence the scores need to be organized to see how an individual 
school is achieving relative to comparable elementary schools. By forming a set of 
peer institutions we can see how an elementary school is achieving in relation to its 
peers. Within a given school there needs to be a way of making comparisons 
among achievement scores for the different academic areas. There is also a need for 
grade level to grade level comparisons. 

For the purpose of this study a set of thirty nine elementary schools within a 
metropolitan area were considered as the set of peer institutions. The mean and 
standard deviation for each subject area and grade level were calculated. Then for 
each school the achievement scores were transformed to z-scores. The z-scores for 
an example school are presented in Table 1. The z-scores are an index as to how 
the mean school achievement scores for this school compare to the other elementary 
schools. A negative z-score indicates the school achievement is below the average 
of other schools and a positive z-score indicates the achievement is above average. 
The z-scores are graphed as a profile in Figure 1. The profile indicates that the 
example school is generally below the average of its peer institutions in 
achievement The profile in Figure 1 is limited in that it does not consider the 
backgounds of the students the school serves. The out of school environment of 
the students which is reflected by their socioeconomic status is a major factor in 
school achievement (White, 1982; Mills, 1983). 
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Consideration of Factors Outside of School Control 



If socioeconomic status of the studenL. is to be considered in the school profile 
there is the issue of how do you measure socioeconomic status. Some of the more 
often used measures consider the educational level of the parents, occupation of the 
parents, family income, value and location of residence (White, 1982). The race of 
students has been identified as a factor associated with achievement by several 
researchers (Dummett, 1984; Matthews 1984; Reyes and Stanic, 1988; Anick 
Clarpenter & Smith, 1981). In an extensive review of previous research White 
(1982) found that socioeconomic measures applied on an individual student basis 
tend to correlate with achievement at a relatively low level. The correlations 
between socioeconomic measures and achievement for aggregates of students tend 
to be relatively high. This implies that if we ate to adjust school profiles for 
socioeconomics it would be best to do it on a xhool basis rather than on an 
individual student basis. 

The aggregate measure of socioeconomics for students in a given elementary school 
can be measured by the percent of students with selected characteristics or 
backgounds. The percent of students participating in freeA'educed lunch programs 
are indication of family income. Mandeville (1988) and Guskey and Kifer (1990) 
found the percent of students receiving free or reduced lunch an effective 
socioeconomic measure in the calculation of SEI's, Mobility as measured by the 
percent to students entering and/or leaving school is a measure of stability of the 
students environment The percent of students living in a two parent household is 
an indicator of the students' home environment. Schools are now required to keep 
records of the ethnic backgrounds of students under the five categories; V/hite, 
Black, Hispanic, Asian and Indian. Because of the current interest in the 
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achievement of minority students the percent of students within the ethnic groups 
was used in the following analysis. 

Table 2 contains a comparison of the example school with its peers on measures of 
socioeconomic status. The socioeconomic status of the students in the example 
school is below the average of its peer institutions. Because of the close 
relationship between socioeconomic status and achievement one would anticipate 
that the example school would have below average achievement 

Another set of considerations for adjustment of the school profile in Figure 1 is the 
resources available to the school. The teaching resource could be measured by the 
average academic preparation and experience of the teachers. Average class sizes 
and available physical facilities could also be considered. The thirty nine 
elementary schools in the peer group are all within the same school district The 
resources have been very carefully allocated so there is very little variability from 
school to school in terms of available resources. Hence the comelations between 
school resources and achievement for the set of peer schools is near zero and was 
not used as a factor in adjusting the schc d 1 profiles. 

Adjustment of Achievement Profiles 

The next step was to look at the relative academic achievement within the school 
with the influence of socioeconomics removed. A regression line for each subject 
and grade level was used to calculate the expected achievement The independent 
variables were percent of students on free or reduced lunch, percent of mobility, 
percent of students living in two parent households, percent of Hispanics, Blacks, 
Asians, and American Indians. 1 hus a set of eighteen multiple regression 
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equations were developed, one for each subject area and grade level. Table 3 
contains the regression coefficients for third grade reading as an example. The 
relative contribution of each of the independent variables is indicated by the 
magnitude of the beta weights. The reader should note, the beta weights for percent 
of minority students are very small in magnitude and thus the percent of minority 
students has very little relationship with school achievement when considered 
simultaneously with the first three independent variables. This is a little 
inconsistent with the findings of previous research projects. This inconsistency 
may be due to the fact that most studies considering the achievement of minority 
students do not account for the socioeconomic factors that tend to be associated 
with minorities. By reviewing test scores of black and white students Burton and 
Jones (1982) found that the difference in achievement of black and white students 
tends to be decreasing in recent years. Lee, Aron and Aron (1988) found that the 
relationship between the percent of black students and achievement may be different 
for schools with a small percent of black students as compared to schools with a 
large percent of black students. 

The adjusted coefficient of determination of .663 in Table 3 indicates that about two 
thirds of the variance in third grade reading achievement among the schools can be 
associated with the seven independent variables. The elementary schools are all in 
the same school district and have very similar instructional programs so the 
coefficient of determination is probably higher than would be expected for a set of 
schools from different school districts. Because such a large portion of the variance 
among the schools can be associated with the socioeconomic factors it is not 
appropriate to compare the school achievements without considering the 
socioeconomic backgrounds of the students. 
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The regression equations were then used to calculate the predicted or expected 
achievement for each school by subject area and grade level. The standardized 
residuals for each subject and grade for the example school are contained in Table 
1. Standardirred residuals have a mean of zero and standard deviation of one. The 
standardized residuals represent the relative achievement of the school by subjea 
and grade after adjusting for the socioeconomic backgrounds of the students and are 
comparable to the SEI's in the studies by Mandeville and Anderson (1987), 
Mandeville (1988) and Guskey and Kifer (1990). Figure 2 contains a graph of the 
standardized residuals. The negative z-scores graphed in Figure 1 indicate the 
school is generally below average in achievement before adjustment Most of the 
standardized residuals are positive indicating that the school achievement is above 
average relative to its peer institutions after adjusting for the socioeconomic 
backgrounds of its students. This is a case of a school serving a low 
socioeconomic group of students and at the same time attaining a higher than 
expected level of achievement considering the backgrounds of the students. 

By comparing the achievement profile in Figure 1 before adjustment and the profile 
in Figure 2 after adjusting for the student backgrounds we can see that achievement 
is a relative thing. There are clearly no absolutes in academic achievement. It is a 
matter of what is the achievement level in relation to the students and resouces of 
the school. 

Interpreting the School Profile 

A closer examination of Figure 2 will give us a picture of the balance of 
achievement within the school. The pattern of the standardized residuals in Table 4 
also gives us another view of the achievement pattern within the school. The 
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school is a little more successful in teaching reading and math than science and 
social studies. A grade level comparison shows that fourth grade is the weakest 
grade level in achievement This gives us some direction as to where to target the 
instructional improvements efforts. Fourth grade is the grade of primary concern 
and the subjects of most concern should be science and social studies. After 
studying several school profiles the author found very few schools have a uniform 
balance in achievement from grade to grade or subject to subject It is fairly easy to 
identify the strong and weak areas of a schools instructional program with an 
achievement profile. 

The primary idea of the school achievement profile is the targeting of instructional 
improvement activities. By attaining a broad overall view of achievement within the 
school the strong and weak areas within the instructional program become apparent. 

Potential Misuse of Achievement Profiles 

It would be very tempting to rank the schools within the peer group based on their 
average standardized residual contained in the bottom right hand comer of Table 4. 
Because of errors in measurement and all of the statistical procedures to generate the 
residuals it is inappropriate to use the profiles to evaluate schools (Guskey & 

Kiefer, 1990). The use of these profiles to evaluate schools would lead to a test 
driven curriculum and the objections of many educators (Biacy, 1987; Madaus, 
1988; Livingston, Castle and Nations, 1989). Because of differences of students 
within the school it is questionable to make grade level comparisons or comparisons 
of students taught by different teachers (Raybom, 1989). 
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Summary 



The statewide testing of students needs to be looked at carefully in terms of the 
overall goals of instruciional improvement. Only by careful organization and 
inteipretation of the test results will we be able to get some clear direction of where 
to place the instructional improvement efforts. The concept of school achievement 
profiles is proposed as another tool to give some direction to instructional 
improvement activities. 
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Table 1. Test Scores for an Example Elemtary School 





Grade 

Level 


School Average 
Raw Score z-Score 


Adjusted Score 
Expected Residual 




2 


335 


.24 


289.0 


1.78 




3 


292 


-1.31 


283.6 


.49 


Reading 


4 


289 


-1.21 


287.0 


.11 




5 


310 


-.06 


285.8 


1.37 




6 


305 


-.70 


292.0 


.80 




2 


331 


.12 


291.6 


1.16 




3 


301 


-1.28 


293.9 


.30 


Math 


4 


270 


-1.61 


278.2 


-.32 




5 


319 


-.33 


297.2 


.83 




6 


373 


.29 


325.1 


2.00 




3 


309 


-1.18 


300.7 


.22 


Science 


4 


283 


-1.43 


309.3 


-.79 




5 


332 


-.12 


308.6 


.70 




6 


370 


-.16 


332.4 


1.24 




3 


320 


OO 

q 

1 


312.9 


.20 


Social 


4 


295 


-1.44 


299.6 


-.23 


Studies 


5 


311 


-.50 


290.9 


.86 




6 


315 


-.67 


304.4 


.42 
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Table 2. Socioeconomic Measures 



Measure 


School 


District Average 


% Free/Reduced Lunch 


67.1 


30.2 


,% Mobility 


82.0 


33.2 


% Two Parent Households 


49.3 


67.4 


% Hispancis 


.0 


.5 


% Blacks 


9.4 


3.4 


% Asians 


.4 


.7 


% Indian 


.8 


.3 
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Table 3. Multiple Regression Coefficients fcM" Third Grade Reading 



Independent Variable 


Correlation 


b 


Beta Weight 


% Free/Reduced Lunch 


-.818 


-.857 


-1.130** 


% Mobiltiy 


-.794 


-.636 


-.727* 


% Two Parents 


.593 


-.509 


-.307 


% Hispanic 


.014 


1.632 


.092 


% Black 


-.393 


.222 


.088 


% Asian 


.251 


.767 


.075 


% Indian 


-.309 


3.139 


.067 



Constant = 413.425 

Coefficient of Determination = .725 

Coefficient of Determination adjusted for shrinkage = .663 

Multiple R = .851** 

Standard Error of Estimate = 17.180 
* p<.05 **p<.01 
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Table 4. Standardized Residuals for the Example School 



Grade 


Reading 


Math 


Science 


Social Studies 


Average 


2 


1.78 


1.61 


- 


- 


1.70 


3 


.49 


.30 


.22 


.20 


.30 


4 


.11 


-.32 


-.79 


-.23 


-.31 


5 


1.37 


.83 


.70 


.86 


.94 


6 


.80 


2.00 


1.24 


.42 


1.12 


Average 


.91 


.88 


.34 


.31 


.64 
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Grade Level 



2 3 4 5 6 




Figure 1 : Example School Profile Before Adjustment 



Grade Level 



2 3 4 5 6 
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Figure 2: Example School Profile After Adjustment 





